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The rapidly advancing capabilities of autonomous aircraft suggest a future where many 
of the responsibilities of today’s pilot transition to the vehicle, transforming the pilot’s job 
into something akin to driving a car or simply being a passenger. Notionally, this transition 
will reduce the specialized skills, training, and attention required of the human user while 
improving safety and performance. However, our experience with highly automated aircraft 
highlights many challenges to this transition including: lack of automation resilience; 
adverse human-automation interaction under stress; and the difficulty of developing 
certification standards and methods of compliance for complex systems performing critical 
functions traditionally performed by the pilot (e.g., sense and avoid vs. see and avoid). 
Recognizing these opportunities and realities, researchers at NASA Langley are developing a 
haptic-multimodal flight control (HFC) system concept that can serve as a bridge between 
today’s state of the art aircraft that are highly automated but have little autonomy and can 
only be operated safely by highly trained experts (i.e., pilots) to a future in which non- 
experts (e.g., drivers) can safely and reliably use autonomous aircraft to perform a variety of 
missions. This paper reviews the motivation and theoretical basis of the HFC system, 
describes its current state of development, and presents results from two pilot-in-the-loop 
simulation studies. These preliminary studies suggest the HFC reshapes human-automation 
interaction in a way well-suited to revolutionary ease-of-use. 
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I. Introduction - A Case for Incremental Revolution 

A common belief is that easy-to-use Personal Air Vehicles (PAVs) can only be achieved through full autonomy. 

Often, the underlying reasoning is that the human simply cannot be trusted to do the right thing. Just looking at 
drivers on the road is enough to confirm this belief. Or is it? While we’ve all seen drivers do crazy things and 
probably made a few blunders ourselves, the bottom line is that we find cars, operated by the public, safe enough 
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that we have made them an integral part of our daily lives. We rarely give safety a second thought when considering 
a car trip. We would all like safer cars and are willing to spend a little more to improve safety, but if people were 
told that the roads were unsafe and that they could not use automobiles anymore until we perfected the autonomous 
vehicle, people would say “Forget that!” and get in their cars anyway. If PAVs could achieve similar ease-of-use, 
utility, economy, and safety as cars, are we not likely to similarly integrate them into our lifestyles, regardless of the 
level of autonomy? 

From practical experience, we are much more comfortable with risk when there is a human in control who can 
respond to novel situations. In all likelihood, a fully -automated PAV would be subject to a much higher standard of 
safety than a piloted one. This added scrutiny would require additional time and money to satisfy. But how much 
time and money will be required, and will the benefit be worth the expense? Assessing the time and costs to bridge 
the technology gap between the current state of the art and full -autonomy within even a rough order-of-magnitude is 
problematic given the required technology breakthroughs. The most fundamental breakthroughs are of course, 
technological substitutes for the perception, world knowledge, general intelligence, and creative thinking that 
today’s pilot brings to the cockpit. As discussed later in the paper, we believe that current, highly automated aircraft 
(manned or unmanned) and experimental, fully-autonomous cars and aircraft give the appearance of being much 
closer to having the necessary, human-like capabilities than they actually are. While current and emerging 
automation often exceeds human performance in terms of isolated engineering criteria, it tends to be inherently 
brittle to even small changes in the task or environment 1 . Despite amazing progress in the development of 
automated vehicles, we still have a long way to go. 

A potentially more challenging problem for aviation applications involving inherent risk to humans (e.g., 
involving human cargo) is certification. The current regulatory framework is predicated on the presence of a human 
in the system, and changing this situation will be a long and contentious process. Current activities to substitute 
“sense and avoid” for “see and avoid” to enable co-mingled manned and unmanned aircraft system (UAS) 
operations illustrate the difficulties of this task and paint a daunting picture for any business expecting a return on 
investment in the near term. On top of these technical and regulatory challenges are societal challenges such as 
public trust and liability. 2010’s rumors about electronics or software causing uncommanded automobile 
accelerations where the driver could not intervene foreshadow possible public concerns. 

Because of these factors, we believe the most promising path, indeed perhaps the only practical path, to PAVs 
retains the human in the pilot’s seat. This approach relies primarily on a restructuring of the Human/ Automation 
Interaction (HAI) experience rather than a technological breakthrough. We believe that this can be achieved with a 
few simple technological advances, using existing technology. This new structure, rather than replacing the human, 
more fully supports them, enhancing their strengths and compensating for their weaknesses. At the same time the 
restructuring will create a framework in which emerging and future technologies can be introduced into the flight 
deck in a managed, incremental process. Initially, new technologies could be used in non-critical roles and later, 
once operational experience and certification criteria are in place, elevated to more critical uses. Through this 
process, we may one day find ourselves with intelligent PAVs that do not require our interaction, and we will have 
enjoyed the benefits of personal air travel along the way. 

But how should the HAI and supporting interface be re-engineered to support these goals? Additionally, will 
pilots perceive enough benefit in terms of safety and ease-of-use, during flight and training, from early 
implementations (e.g., retrofit) to begin and sustain a process of evolution that results in a true revolution? 
Presently, we have only partial answers to these questions. This paper provides a review and update on one of our 
concepts: a variable autonomy, haptic -multimodal flight control (HFC) system. The term “haptic” is used here to 
refer to the use of touch, force, and motion to underlie the HAI. The HFC is intended to be a component of an 
overall flight deck 2 and provides integrated support for control and tactical trajectory management (CTTM) tasks. 
These tasks include, guidance, navigation, and control; formulation of trajectories and maneuvers satisfying near- 
term objectives and constraints; and tactical (i.e., time -constrained) decision making in potentially dynamic 
situations. Examples of tactical decision making include deciding whether to continue or abort an approach; 
whether to accept a proposed clearance; how to reroute around traffic or local weather; and how to get on the ground 
safely in the event of a fire. In brief, CTTM can be thought of as the “aviate task” and time -pressured portions of 
the “navigate task” of the familiar “aviate, navigate, and communicate” prioritization scheme 3 . CTTM is central to 
safe and effective flight and breakdowns within CTTM, typically loss-of-control or/and poor tactical decision 
making, are causal or contributing factors in a large majority of aviation accidents 4, 5 . 

In the remainder of the paper we describe the expected requirements for PAV ease-of-use, the current state of 
the art in aviation and experimental autonomous vehicles. Next we describe the design concept and functionality of 
the HFC. We then summarize the results from two pilot in the loop simulation studies supporting the expected 
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human-automation interaction benefits. We conclude with a summary and brief discussion of future efforts needed 
to enable PAVs. 

We use the term “pilot” throughout the paper to broadly refer to the human presence in the human-aircraft 
system and it may, particularly when referring to present day operations, reflect multiple human actors (e.g., a two- 
pilot crew and ground-based dispatcher). Similarly, when the term “pilot” is used to refer to an expert with a high 
level of preparation and training, this usage will be specifically noted. Also, while concepts of the HFC can be 
applied to unmanned aircraft, development to date has focused on on-board pilots and this focus is reflected in the 
current paper. 


II. Background 


A. Personal Air Vehicle Requirements 

The expected operational characteristics and piloting requirements for PAVs are summarized in this section. 
The reader should recognize that given the notional nature of PAVs at this time, these characteristics are more 
illustrative than prescriptive. PAVs are expected to be light-aircraft carrying 1-10 people, including a pilot. 
Configurations may range from simple, single-engine piston aircraft up to very light jets. In the future, operations 
from small urban and suburban airfields may require precise, short or vertical takeoff and landing operations 6 . 
Flight in (near) all weather conditions is required to support reliable transportation. Early PAVs will have to operate 
in the National Airspace System (NAS) according to current and emerging rules such as being developed for the 
“NextGen” NAS 7 . Longer term, widespread adoption of PAVs would result in air traffic densities much greater 
than envisioned for NextGen and separation intervals in low- visibility on the order of 30 seconds (~0.5 nm) may be 
needed in the vicinity of airports for adequate runway throughput. Propulsion systems are assumed to be controllable 
by something like a single-lever power controller with a full -authority, inner-loop engine controller. Loss of power 
is expected to be improbable, but something that needs to be considered within the context of CTTM. In-flight 
recovery systems such as a whole-aircraft parachute may be present, but they should be given no credit as a means 
of reducing the certitude of other systems. 

Single-pilot operations are the norm for PAVs with ratings for non-commercial (i.e., private pilots) and 
commercial (e.g., air-taxi pilots) operations. Initially, these pilot ratings may fall under current pilot licensing 
requirements, albeit with equipage restrictions similar to a centerline -thrust restricted, multi-engine rating. With 
improved technology and operational experience, training required for near-all weather operation might be reduced 
to a level comparable to that of driver education (e.g., 24 hours of classroom (or on-line) instruction and 40 hours of 
supervised operation). This total is similar to the minimum hours to earn a visual flight rules, private -pilot license 
today may raise concern given the low completion rate of current student-pilots, which is around 20% 8 . We believe 
the envisioned PAV rating would result in a much higher degree of transportation utility and result in completion 
rates similar to driver education. In addition to simplified initial training, another goal is to make piloting easy 
enough that flying solely to maintain proficiency is unnecessary for pilots flying once a month for transportation 
purposes. Finally, the risk of a serious accident should be no greater than travel by car. While it can be difficult to 
directly compare automotive and aviation safety statistics given the concentration of risk around takeoff and landing, 
we believe this goal will be achieved if the PAV fatal accident rate is an order-of-magnitude less than today’s small 
aircraft. The current small aircraft fatal accident rate is 1.3 per 100,000 flight hours 9 so a target rate of 1 per 
1,000,000 flight hours would be appropriate. 

In the next section, we review current highly automated aircraft such as modern jets and many new small aircraft 
as well as experimental “driver-less” vehicles. 

B. Assessment of the State-of-the-Art 

How close is the current and emerging state of the art to enabling ease-of-use? In the next few paragraphs, we 
briefly review the capabilities of contemporary, highly-automated aircraft (cHAA) and experimental, autonomous 
vehicles with an eye toward the technological foundation they provide and insights they offer into HAL 

The terminology surrounding automated and autonomous vehicles can be confusing because terms like fully- 
automated and autonomous are often used interchangeably but do not necessarily mean the same thing. For example 
the Global Hawk unmanned aircraft is often referred to as “autonomous” because it can conduct entire flights 
without interacting with a ground operator. However, a more appropriate description might be “fully-automated” 
rather than autonomous given that it follows pre- scripted mission plans, which can take up to four weeks to program 
and validate 10 , and is vulnerable to unforeseen but not improbable hazards like a truck parked on the runway. For 
practical purposes what is usually meant by an autonomous vehicle is a vehicle having a level of intelligence, i.e., 
the ability to make appropriate choices or decisions in an uncertain environment that is sufficient to perform 

3 

American Institute of Aeronautics and Astronautics 



complex missions without a human operator or mission-specific low-level programming defining all actions. This is 
the general meaning we associate with autonomy in the following discussion. We use fully-autonomous to indicate 
a vehicle needing no human interaction. The reader will also notice that the following discussion is largely silent on 
fully-automated aircraft like the Global Hawk. In general, these systems suffer from similar HAI challenges as 
piloted cHAA, often with more serious consequences given the great reliance on automation and separation between 
the operator and the vehicle 11 . 

Contemporary highly-automated aircraft including modern transports, business jets, and many small aircraft now 
have sophisticated automation and can perform routine flights with little pilot interaction. But, it is important to 
recognize that this capability is far short of what is required for fully -automated, passenger carrying PAVs and is 
very different from the pilot playing a minor, easily eliminated role or requiring significantly less training, practice, 
or vigilance, than pilots of less automated aircraft. As Billings states, 

The newest long-range airplanes can operate almost unassisted from shortly after takeoff in New York until coming to 
rest after a landing in Tokyo. The considerable psychomotor and cognitive skills of their human operators are hardly 
called upon unless some element of the automation fails or unanticipated environmental circumstances arise. But when 
the environment does not behave as expected, or when the very reliable machinery does not function correctly, we expect 
these human operators to do whatever is required to complete the mission safely . 1 

In effect, today’s aircraft can perform the deterministic, routine and repetitive tasks that make up the most 
recognizable aspects of aircraft operation, but the most challenging tasks, including detecting and recovering from 
automation failures, are left up to the pilot. From a training and safety perspective, any task that a pilot may be 
called on to perform, even if only rarely, is still a task that they must be trained and prepared to perform. This 
required training does little to improve ease-of-use and introduces safety concerns due to potential lack of 
proficiency. The practice of automating routine tasks and leaving the more difficult leftovers to the pilot leaves a 
large gap to be crossed before fully autonomous PAVs can be realized. For piloted PAVs, operational experiences 
with cHAA provide insights into HAI constraints that must be addressed in the PAV’s pilot interface; we discuss 
these issues in a latter section. 

If current, conventional automation is a long way from enabling autonomous PAVs, a good question is: how 
quickly the technology from experimental, autonomous vehicles might close this gap? Research to expand the 
capabilities of fully-autonomous vehicles is receiving an enormous amount of attention and investment. The 
plummeting cost and increasing performance of sensors and computing hardware make this a fertile area for 
research. The most active area of research has probably been autonomous cars given the large number of 
applications, potential safety benefits, and visibility gained from a series of “Grand Challenges” sponsored by the 
Defense Advanced Research Project Agency (DARPA) between 2004 and 2007 12 . In the most recent event, the 
Urban Challenge held in 2007, autonomous cars had to navigate through an urban environment including the ability 
to perceive and interact with other traffic. The winning entrant completed a 60 mile course with an average speed of 
14 mph. Based in part on this work, Google recently launched a research program to develop autonomous cars 13 . 
Google’s motivations include improved safety (targeting a 50% reduction in fatalities), and the transformative 
impact self-driving cars would have on car-sharing. So far, Google’s cars have logged over 140,000 miles without 
an accident, although always with a trained safety driver and led by a conventionally-driven car to ensure the route 
map is accurate and road conditions satisfactory. Comments from knowledgeable demonstration passengers, such as 
the adminstrator of the National Highway Traffic Safety and chairwoman of National Transportation Safety Board 
Chairwoman, find the technology impressive albeit with a long way to go before widespread, public use 14 . 

The developments taking place on these vehicles and similar experimental flight vehicles 15 are impressive and 
important advances that will improve the safety of ground and flight vehicles, both manned and unmanned. But, 
from the perspective of PAVs, what should be their role in the near future? What these activities most clearly 
demonstrate is that current technology is readily available to perform many of the low level tasks in the vehicle (e.g., 
staying in a lane, not following too closely, maintaining speed, staying on course) and can do so far better than a 
human operator. The performance areas that technology is reaching for and most likely to fail at (and indeed is most 
impressive when it comes close to performing) are those having to do with higher level, common sense, adaptive, 
and creative troubleshooting performance. Here, even the best technology is highly vulnerable. But we have those 
skills in abundance in the form of the human. Why do we feel that we must ignore those skills in the name of ease- 
of-use and reliability? Yes, humans can and will make mistakes, but can we provide countermeasures for their error- 
proneness while utilizing their skills and still make flying easy and enjoyable to fly? We strongly believe that the 
answer is ‘yes.’ And no Grand Challenge is needed to prove it. 

If current automation and emerging autonomous vehicle technologies are not close to bridging the gap to fully- 
autonomous PAVs, we are compelled to develop a more effective human-machine partnership. To do this, we take 
advantage of the insights provided by cHAA. 
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C. Where Humans Fail: Insights from Contemporary Highly Automated Aircraft 

While advanced avionics (e.g., flight management systems, auto-flight systems, electronically controlled 
propulsion, and advanced subsystems management) have greatly improved aviation safety, there are two important 
insights to be gleaned from their introduction and use. The first insight is the fallacy of reduced workload. As early 
as the 1990s, research showed that highly automated flight decks did not necessarily reduce crew workload. In fact, 
workload was reduced in already low periods but could be dramatically increased in high- workload periods 1 . This 
situation was due to automation that handles routine situations but generally gives up in uncertain, novel, or 
complicated situations. As far as workload is concerned, automation does not appear to be helping when it is needed 
most. Additionally, the presence of automation in the form of the flight management computer adds additional tasks 
(higher workload) just to program and remember how to operate the automation. 

The second insight comes from the types of errors that humans make that currently lead to accidents. Figure 1 
from Wiegmann and Rantanen 16 shows the relative frequency of different classes of human error in commercial and 
general aviation respectively, giving insight to areas of human weaknesses. Wiegmann and Rantanen describe the 
error categories as follows: 

Skill-based errors : Highly practiced behavior that occurs with little or no conscious thought. These “doing” 
errors frequently appear as breakdown in visual scan patterns, inadvertent activation/deactivation of switches, 
forgotten intentions, and omitted items in checklists. 

Decision errors: These “thinking” errors represent conscious, 
goal-intended behavior that proceeds as designed, yet the plan 
proves inadequate or inappropriate for the situation. These errors 
typically manifest as poorly executed procedures, improper 
choices, or simply the misinterpretation and/or misuse of 
relevant information. 

Perceptual errors: These errors arise when sensory input is 
degraded, as is often the case when flying at night, in poor 
weather, or in otherwise visually impoverished environments. 

Faced with acting on imperfect or incomplete information, 
aircrews run the risk of misjudging distances, altitude, and 
descent rates, as well as of responding incorrectly to a variety of 
visual/vestibular illusions. 

Violations: Violations are divided into two sub-categories, 
routine and exceptional. Routine violations, often referred to as 
“bending the rules,” tend to be habitual by nature and are often 
enabled by a system of supervision and management that 
tolerates such departures from the rules. Exceptional violations 
are isolated departures from authority, neither typical of the 
individual nor condoned by management. 

What is important to note is that the errors are not being 
made in the areas emphasized in research into fully autonomous 
vehicles. They are being made in areas that machines are 
traditionally very proficient, i.e., precise, skill-based operations. 

Automation can keep an aircraft stable for the most part. There 
are certainly cases where the automation cannot, based on its 
limited authority (e.g., severe icing), but what is needed is the 
ability to alert the human to an approaching limit long before a 
problem occurs and then let the human make a decision 
regarding what to do. 

D. Current Small Aircraft Safety and Ease-of-Use 

Starting from the premise that neither the prospective PAV pilot nor automation, acting alone, can perform the 
complete task, our goal is to develop a joint human-automation system that can. The requirements are, in part, that 
the system be safe and easy to use in both initial training and operation. In this section, we briefly review current 
small aircraft safety and training to identify the feasibility of meeting the goals mentioned earlier (i.e., an order-of- 
magnitude improvement in the fatal accident rate, and enabling safe, confident all weather operations with 
approximately 40 hours of training). We also use this review to indentify the key factors in achieving these goals. 


Commercial Aviation 



Perceptual Violations- 
Errors 



FcrcepcuaJ Violations 
Errois 


Figure 1. Percentage of commercial and 
general aviation accidents associated with 
various human error causal factors 16 
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Figure 2. General aviation accident first occurrences, 2006 
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Figure 2 from the National Transportation 
Safety Board 17 shows the first occurrences 
leading to accidents and fatalities for small 
aircraft in 2006 (the most recently published 
data). The first occurrence can be considered 
the first observable link in the accident chain of 
events. Preventing loss of control and avoiding 
hazardous weather, terrain, obstructions, and 
traffic would eliminate approximately 80% of 
current fatal accidents. The majority of 
remaining accidents would then be attributable 
to loss of power. Given the older engine 
technology represented in this 2006 data (e.g., 
the average age of single-engine aircraft in the 
United States is greater than 30 years), it is 
reasonable to expect that modern turbine or 
electronic engines on PAVs could reduce these 
events by 50% or more. Together, these items 
provide a potential order-of-magnitude reduction 
in fatal accidents relative to current operations. 
If achieved, this would make travel by personal 
air vehicle as safe as driving by car. Current 
technology in the form of terrain and obstruction 
databases, in-flight weather, cockpit display of 
traffic, and the ability to display and overlay 
these items on a multi-function display already 
provides a solid foundation for hazard avoidance. The key outstanding challenge to achieving the desired increase 
in level of safety is preventing loss of control. 

From a training perspective, fig. 3 presents the approximate percentage of time spent learning various skills to 
become an instrument-rated private pilot. The total time required to reach this point is approximately 150 hours of 
flying, consisting of a mix dual instruction and solo practice. This data, collected by Embry Riddle Aeronautical 
University in the 1990’s 18 , is somewhat dated as it predates the widespread use of the Global Positioning System, 
introduction of glass-cockpit airplanes, and updated instructional methods such as scenario -based training. The 
most significant component of training is the amount of effort spent becoming proficient at basic flying skills, i.e., 
learning how to manipulate the stick, rudder, and throttle to make the airplane go where the pilot wants it with the 
right attitude and energy, and knowing how it should be flown and positioned for routine operations and select 
emergencies such as off-field landings. These tasks would be minimally affected by the introduction of glass 
displays and still require significant attention to master. 

Interestingly, multiple simulation studies have 
investigated the potential benefits of using pathway- 
based displays to intuitively indicate where the aircraft 
should be flown combined with fly-by-wire (FBW) 
control that dramatically simplifies manual control 19, 20 . 

These studies uniformly show that using automatic 
control to make the pilot’s manual control inceptors 
(i.e., stick and throttle) directly command the velocity- 
vector of the aircraft allows a non-pilot to perform 
precision instrument maneuvers, takeoffs, landings with 
only minutes of training. Figure 4 illustrates how 
quickly a non-pilot learns to control an airplane through 
a velocity-vector-based control system. The figure 
shows lateral tracking error for a untrained, non-pilot 
flying an extended airport traffic pattern with 
conventional aircraft dynamics and a simplified, 
velocity- vector-based dynamics. After overshooting in 
the first turn ( at ~60 seconds) the subject maintains 



Figure 3. Components of flight training, ab initio to 
instrument rating. Total-time is approximately 150 
flight hours 
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lateral tracking within +/- 250 feet for the rest of 
the run, even during 90 degree turns. In 
comparison, the same subject still has trouble 
controlling the lateral trajectory at the end of the 
run with the conventional controller, blundering 
approximately a Va mile off course. In Stewart’s 
study 20 , the participants took off, flew a race track 
pattern around the runway, and then landed. The 
task was performed without external visibility 
above 200’. Of the 7 novice subjects tested, all 
completed the task on the first run without prior 
training or practice. After 10 runs all subjects 
successfully completed the task in the presence of 
simulated moderate turbulence and a steady 20 
knot wind, quartering the runway. If these 
Figure 4. Comparison of conventional and simplified simulator results could be realized in the field 
control for non-pilot test subject 19 (admittedly a big if), prospective PAV pilots could 

perform unassisted takeoffs, landings and pattern 
operations during their first lesson. Rather than extensive training and practicing of cross wind landings, slow flight, 
turns around a point, basic instrument flight, and practice approaches to internalize the low-level perceptual motor 
skills, more training could focus on the higher-level judgment and decision making important to non-normal 
situations. Another significant opportunity for reduced training, and simplified operations, is streamlining the 
number of procedures in use as experience is gained with Required Navigation Performance (RNP) and NextGen 
operations. For example, presently there are numerous instrument navigation aids and associated approach 
procedures in use (e.g., NDB, VOR, LOC, VOR/DME, RNAV, LNAV/VNAV, LPV, ILS, ILS Backcourse, RNP) 
and instrument-rated pilots need to be prepared to use some number of them. In contrast, RNP navigation is 
independent of specific navigation sensors and when mature should allow a single, standardized approach type to be 
used to all runway ends. Similarly, NextGen, or more likely its descendents, may streamline air traffic management 
and traffic separation procedures so that difficult, time-pressured voice communication fades away and is replaced 
by something less burdensome to the pilot and that also simplifies the coordination of very high-density operations. 
At some point, separate visual and instrument flight rules may disappear and be replaced by a single set of rules used 
regardless of external visibility. 

Finally, in the NextGen environment, information needed for making decisions will be generally more complete 
and easier to obtain compared to today’s environment which relies on multiple, dissimilar, sources on the ground 
and in flight. These improvements will reduce the instruction and practice spent learning how to obtain information 
pertinent to a flight and allow more training on interpretation and decision making. From the perspective of CTTM, 
the awareness of routine hazards (e.g. traffic, terrain, obstructions, weather) and factors of flight (e.g., local airport 
status include runways in use, pattern direction and winds) afforded by the automation is expected to be nearly 
complete. 

Given the available data, it is premature to forecast a minimum training time for PAV pilots. However, in 
reviewing current flight training requirements and predicting the affects of available and emerging technologies, a 
long-term goal on the order of 40 hours appears to be credible. In the next section, we provide an overview of the 
HFC concept that seeks to address the safety, ease-of-use, and HAI issues just reviewed. 

III. Haptic-Multimodal Flight Control System 



A. HFC Description 

This section provides an overview of our proposed HFC system for safe, simple and effective CTTM for PAVs. 
As mentioned in the introduction, the HFC is part of an overall flight deck concept known as the “Naturalistic Flight 
Deck” (NFD) 2 . As described in detail in reference 1, the inspiration of the HFC is the metaphor of a rider and good, 
well trained horse. The reader is encouraged to read Flemisch, et al. 21 for a detailed description of the metaphor as 
only a brief description is provided here. Key aspects of the metaphor for design are to provide: 

• A vehicle with limited, specialized intelligence, sufficient to move safely and appropriately through the 
environment without human oversight in routine situations 

• Shared control with an operator facilitated through haptic -multimodal interaction. 
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As applied to aviation, the first 
aspect suggests an aircraft with the 
ability to maneuver through airspace 
without endangering itself (e.g., staying 
within envelope limits, avoiding 
hazards) or others. More advanced 
implementations could even recognize 
runways (i.e., perceiving boundaries and 
obstacles rather than relying entirely on 
a priori data) and autonomously takeoff 
and land, albeit usually at the command 
and supervision of the on-board pilot. It 
also suggests some degree of 
independent will if moving toward a 
self-perceived threat. Limited 

intelligence implies that the aircraft, like 
a horse, knows the near-term actions 
desired by the pilot but does not know 
about their long-term intentions (other 
aspects of the NFD support longer-term, 



Figure 5. Automation spectrum with the two levels of support, 
Tight-rein and Loose-rein (above) with different influence of pilot 
and automation on the aircraft behavior (below) 22 


flight management tasks). The pilot uses simple interactions to keep the aircraft on the desired path when significant 
trajectory changes are required. If the pilot misses an expected action, other aspects of the NFD support pilot 
monitoring, alerting, and reengagement. In an advanced implementation, if the aircraft perceives an imminent threat 
and the pilot is not engaged, it should transition to a more cautious posture, shifting its bias toward safety. 
Ultimately, it would perform a precautionary, autonomous landing at some point if the pilot remained non- 
responsive. This situation should probably be considered an emergency, allowing special handling in the airspace 
and discouraging abuse of this back-up capability. 

The concept of shared control implies that neither the pilot nor vehicle can hand over all aspects of the CTTM to 
the other agent. Currently, there are two primary modes of operation known as “tight” and “loose” reins. As shown 
in fig. 5 22 , tight-rein corresponds to a form of assisted manual control and loose-rein a form of highly-automated 
control. As explained shortly, the assisted manual control is provided by a flight -critical fly-by-wire system that 
supports all other control modes. In tight-rein, higher-level automation functions nominally remain passive, but if 
hazards are approached they generate forces (i.e. pushes back) on the pilot’s hand controls consistent with a 
preferred avoidance maneuver. 

The highly-automated control mode differs from cHAA in two fundamental ways. The first, as already 
mentioned, is that the automation cannot be coupled to an entire flight plan and requires pilot involvement at 
significant junctions. The second is that all automation support is managed through haptic and visual elements 
integrated into the primary control interfaces (i.e., force-feedback stick and throttle) and the displays (a synthetic 
vision based primary flight display (PFD) and tactical navigation display (TND)). Both of these features have been 
evaluated in part-task simulation studies 

H-lnspired Outer Loop 
Automation 


summarized in an 


and the results are 
upcoming section. 

A simplified system diagram of the 
HFC system is shown in fig. 6. The 
inner-loop control simplifies the 
dynamic response of the aircraft, 
providing predictable control and 
envelope protection for both the pilot 
and outer-loop automation. As 

described in the earlier safety and 
training overview, simplified flight 
dynamics combined with envelope 
protection are central factors in 
achieving improved safety and ease-of- 
use. Implementing these functions is 
probably best done through a FBW 
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Figure 6. System diagram of HFC 
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control system. While FBW systems are certified on current civil aircraft, these implementations all have failure 
modes that revert to the pilot essentially flying the bare airframe. This sort of reversion is, of course, unacceptable 
for a FBW PAV. Developing a certifiable FBW system that continues to provide simplified dynamics and 
protection against all failures that are not “extremely improbable” is a major technical challenge of our concept. 
That noted, this task should be far easier than developing and certifying a fully autonomous system. It should also 
be noted that many of the benefits of the HFC in terms of improved human -automation interaction and envelope 
protection are possible on non-FBW aircraft, albeit with increased training because of the more complex flight 
dynamics. 

The active, force-feedback control interfaces, represented by the control stick in fig. 6, provide the central point 
of contact between the pilot and automation. In our current implementation, displacement of these interfaces is the 
sole connection between the outer- (i.e., assessment, path planning and autopilot) and inner-loop (stability 
augmentation) control systems. The practical effect of this implementation is that both the pilot and automation 
have to apply forces to the stick to affect maneuvers. If only one agent applies force, the resulting stick 
displacement provides observable kinematic feedback to the other. If both agents apply force, differing intentions 
can be intuitively felt through the proprioceptive interaction. The immediate response of the airplane is determined 
by the control displacement resulting from the net force input. By increasing or decreasing the forces used by the 
HFC, its “strength of will” can be modulated between suggestive and dominant. When using light or moderate 
forces, the pilot can overpower the automation and retain command authority. As described below, the HFC uses its 
sensing of pilot-applied forces, combined with simple interpretations of the flight situation, to modify or change its 
actions to better support the pilot. 

The outer-loop or automatic guidance functions of the HFC are activated and managed through the primary 
controls and displays in a manner that leverages the skills and interactions the pilot uses for manual flight. A more 
detailed explanation is available in ref. 23, but the general principles are summarized here, as are some extensions 
made since ref. 23 was published. The PFD and TND (fig. 7) display spatial representations of the external 
environment, overlaid by basic flight parameters. The depiction of the environment includes physical elements like 
terrain, obstructions, weather, and traffic. The depiction also includes virtual elements like published airspace 
features (e.g., fixes and airways) and, if present, the planned route of flight. The pilot can direct the HFC to couple 
to one of these displayed paths using a set of simple interactions described in the next few paragraphs. 

If the pilot is free maneuvering (i.e., flying in tight-rein) and wants to direct the airplane to join one of the 
displayed paths (i.e., transitioning to loose-rein) he can do this two ways. The first method, called “Point to 
Program” (P2P), is shown in fig. 7. To perform a P2P coupling, the pilot maneuvers the airplane so it is aimed at the 
desired intercept point. The HFC indicates its recognition of situation by highlighting the targeted segment (fig. 7a). 
If the pilot pulls the trigger on the stick while the HFC is in this armed state, the HFC generates, displays, and 
couples to an intercept path onto the targeted segment (fig. 7b). A second button on the stick is used to cycle 
through and select the desired path if more than one path falls within the targeting criteria as shown in figure 7a. 
The second method of joining a displayed path is to manually maneuver into proximity and alignment with it. The 
HFC recognizes this geometry and couples automatically. 

Leaving a path and returning to tight-rein is done by either making a sustained maneuver away from the path or 
pressing a third button the stick. Once established on the path, along-path trajectory junctions (i.e., forks in the 
road) can be selected using the same button used earlier to select from multiple P2P options. The aircraft’s altitude 
can be managed independently from the nominally displayed path altitudes on the PFD. This is done using a hat 
switch on the stick to create and set a “designated altitude”. With a designated altitude active, offsets of 
situationally appropriate paths are displayed at this altitude and available to the HFC. The pilot must manually 
initiate the climb or descent toward the offset, but the HFC levels off when the target altitude is reached. During this 
transition, the HFC continues to follow the lateral definition of the path. 

Functionality similar to an airline autopilot mode-control panel is supported using a second type of path know as 
a “track-hold.” The HFC generates, displays, and couples to track-hold mode if the pilot pulls the trigger to initiate 
a loose-rein transition when no path is targeted for a P2P. The HFC creates the track hold along a linear extension 
of the aircraft’s velocity vector when the trigger is pulled. The resulting path maintains a constant ground track and 
inertial flight path angle. Designated altitudes can be used to manage altitude while traveling along a track hold as 
described earlier. When operating in track-hold mode, the HFC monitors pilot-applied forces. Larger force inputs 
cause the HFC to transition back to tight-rein. Moderate pilot forces cause the HFC to update the heading and angle 
of the track hold as the aircraft maneuvers. This works similar to airline autopilot “control wheel steering,” allowing 
the pilot to make path adjustments without having to go back into tight-rein. It is also possible to use P2P intercepts 
from a track-hold to join displayed segments. 
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The haptic power lever allows integrated management of power, speed, and temporal functions through this 
single interface. As currently implemented, the angle of the power lever represents the power (or thrust) command 
to the engine controller. By pulling the trigger on the throttle, the HFC activates an auto-throttle mode, back driving 
the power lever as needed to maintain commanded airspeed. When first activated, the commanded speed is the 
current airspeed. The commanded value is updated when the pilot applies force to the power lever. The airspeed 
command is changed at a rate proportional to the pilot’s force. 

Higher-level, speed-related modes can be managed using switchology similar to what was described for the 
stick. For example, when coupled to a path, a time of arrival control mode can be armed via a second button press 
on the power lever. From this condition, a hat switch is used to step-through designated, along-path, metering fixes. 
If the pilot pulls the trigger on the power lever from this armed condition, the time of arrival control mode engages, 



Figure 7a. Displays just prior to performing P2P to join a published approach segment 



Figure 7b. Displays just after completing P2P 
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using the current estimated time of arrival to the fix as its target. Subsequent moderate pilot forces on the power 
lever are used to change the target time of arrival in a manner similar to managing airspeed. 

In the next section, we summarize the results from two preliminary, pilot-in-the-loop simulation studies 
evaluating the effects of the HFC on task performance, workload, and awareness. 

B. Initial Human-in-the-Loop Evaluations 

This section summarizes some of the key findings from two initial evaluations of the HFC in piloted simulation 
studies. The first study evaluated the usability of the HFC in still and turbulent air. The second study assessed pilot 
acceptance of the HFC compared to that of conventional interactions. It also assessed HFC’s influence on pilot 
workload, situation awareness and task performance compared to conventional interactions. In contrast to the theme 
of the paper, we’ve used moderately experienced, general aviation pilots as subjects rather than non-pilots in these 
studies. There are several reasons for this. First is the reduced training and improved consistency of subjects who 
are already familiar with flying. Second, these subjects are familiar with current technology and can provide 
comparative feedback based on their experiences. Finally, the initial application of many concepts from the HFC is 
likely to be on current light aircraft rather than PA Vs. 

1. Usability Assessment 

The first study 2 * * * * * * * * * * * * * * * * * * * * 23 evaluated the basic usability of the HFC in calm and turbulent conditions. A particular concern 
was whether turbulence would make the P2P and proximity -join concepts impractical. To evaluate this possibility 
subjects exercised the elemental automation management tasks supported by the HFC in three conditions in a 
motion-based simulator at NASA Langley. The three conditions were: 1) cockpit motion on, turbulence off; 2) 
cockpit motion off, turbulence on; 3) cockpit motion on, turbulence on. The fourth possible combination, motion 
and turbulence both off was found to be equivalent to condition 1 during dry runs and was not included in the test in 
the interest of efficiency. Condition 2 is of interest to unmanned applications during which the vehicle could be 
subjected to turbulence but not the operator. Because the task elements effectively involve manual control tasks, 
Cooper-Harper Ratings (CHR) 24 were found to be an appropriate and effective tool for assessing subjective 
performance and workload. Figure 8 shows a subset of the CHR results from ref. 23. The overall results of this 
study indicate that the five test subjects generally found the HFC easy to use. In turbulence (with and without 
motion), the handling qualities ratings typically increased slightly (probably indicating higher workload), which is to 
be expected. Subjectively, the subjects did not consider this increase to be objectionable and generally less than 
might be expected with conventional automation interfaces in similar turbulence. All five subjects rated the 
difficulty of learning to use the HFC as either easy or very easy. It should also be noted that after this study was 
conducted, significant refinements have been made to further improve the usability of the HFC. 

2. Evaluation of Acceptance and Comparison with Conventional Automation 

The second experiment evaluated pilot acceptance of the HFC concept relative to cases with “conventional” 

automation and with no automation. The study also assessed the influence of these automation levels on pilot 

workload and situation awareness. The results of this study will be documented in more detail in a future report and 

are summarized here. 

At first glance, the haptic, single-point-of-control concept can appear to be a technological step backwards. In a 

modern flight deck, the pilot rarely has to touch the control inceptor. The use of the control inceptor to fly the 

aircraft is virtually the hallmark of unautomated aircraft. Won’t operators consider this to be a step in the wrong 

direction? After all, use of a manual inceptor may increase their workload in some situations. 

We performed an experiment that tested whether pilots preferred the more highly automated approach. In this 

experiment, 24 instrument-rated pilots with little or no glass cockpit experience but fairly high number of flying 

hours were tested. Subjects were asked to plan a route in a fictitious airspace, enter that route into a graphical flight 

planner (virtually connecting the waypoints from point A to point B), and then to fly that route (based on guidance 

from the graphical flight planner) in a desktop simulator. Each pilot experienced three flight control conditions: 

Manual (where they flew the simulator using the side stick with no support from automation), Automated (where the 

aircraft flew itself based on the plan in the planner), and a subset of HFC called Segment-Based Automation (where 

the subject used the side stick to maneuver from one published leg to the next; after coupling onto that leg, the HFC 

automation flew the aircraft along that segment.) 

All three conditions had the same guidance and symbology on the PFD and TND. In all conditions, as the 

aircraft approached a segment change, the subject received a ‘rumble strip’ vibration in the side-stick and an aural 
alert. While flying the routes, subjects were given a number of air traffic control-like instructions that required them 

to fly a parallel path, make a strategic route change, and change runways while on approach. While flying the route, 
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Figure 8. Handling qualities ratings of HFC elemental tasks 

the subjects also had two secondary tasks - a self paced math task and a target recognition task. They were told that 
the flying task was their primary task. 

After experiencing all three conditions, subjects were asked to rate their most preferred control mode and their 
least preferred. They were asked to consider their preference with regard to all three tasks together and to just the 
flying task. The results showed that pilots overwhelmingly preferred the Segment-Based Automation over the 
Manual and the Automated conditions. This was true for both the flying task by itself (fig. 9a) and the combined set 
of tasks (fig. 9b). These results were statistically significant (p < 0.01, where p is the probability of obtaining the 
observed effect under a ‘null hypothesis’). What was also interesting was that their responses to the least preferred 
method of control were split between the Manual condition and the Automated condition (fig. 10). This indicates 
that neither the Automated nor the Manual is a clear second place favorite. A fully automated flight deck would not 
be preferred by the majority of the subjects. 

Additionally, subjects were asked to rate their own situation awareness for the three different control conditions. 
Again, they were asked to consider just their flight situation awareness (e.g., where they were, how fast they were 
going) and their overall situation awareness (e.g., flight plus how many questions they had answered, where the last 
target was). Again, the results significantly favored the Segment-Based Automation over the other two (p < 0.05) for 
both the flying task (fig. 11a) and the combined task (fig. 1 lb). Similar to the preference questions, the results were 
split regarding which condition they felt the least aware of their situation (fig. 12). An attempt was made to 
objectively validate the subjects’ ratings of their own situation awareness. A Situation Awareness Global 
Assessment Technique (SAGAT) test battery 25 was developed that probed the subject at random points in the test to 
ask them situation awareness questions with all the screens blanked out. The probes consisted of asking them two 
flying questions (e.g., what’s the next waypoint? What’s your current airspeed? Where are you on the map?) and one 
target question (e.g., What was the color of the last target? Where was the last target?) and one math question (e.g., 
how many questions have you answered, what was the last answer you typed in?). There were three SAGAT probes 

12 

American Institute of Aeronautics and Astronautics 



during each run and they always asked different questions. The results were not significant. However the numbers 
were higher for the Segment-Based Automation than the other two. It may be that due to the small number of 
questions, the SAG AT was simply not sensitive enough to produce a significant result. Figure 14 shows the results 
for both the flight information and the combined information. 



Figure 9a. Most preferred for the flying task Figure 9b. Most preferred for the combined tasks 
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Figure 10. Least preferred conditions 
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Figure 11a. Highest situation awareness regarding Figure lib. Highest situation awareness regarding 
Flying combined tasks 
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Figure 12. Least situation awareness 
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Figure 13. SAGAT results 


14 

American Institute of Aeronautics and Astronautics 



Figure 14. Highest and lowest (i.e., least) workloads 
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Figure 15. Task Load Index results 


Finally, subjects were asked which condition had the least and the greatest workload. The results are not 
surprising (fig. 14): They rated the Automated as having the lowest workload and Manual as the highest (p < 0.01). 
What is interesting is that the Segment-Based Automation never rated as highest or lowest. We believe that this 
represents a ‘Goldilocks’ spot with regard to workload - “not too much, not too little, just right.” We also used the 
NASA-TLX workload measurement instrument 26 . Statistical significance varied (e.g., for Mental and Effort 
Workload, the Manual was significantly higher than the Fully Automated or Segment-Based Automation (p < 0.05), 
but no significant difference between the Segment-Based Automation and Fully Automated; and there was no 
significance in the differences other differences.) However, the raw TLX data (fig. 15) taken in conjunction with the 
“greatest and least” data sets further reinforces the ‘Goldilocks’ claim. 

The results of this study indicate that Segment-Based Automation using the HFC concept is not a step 
backwards, but a significant step forwards. 
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III. Conclusion 

Personal air vehicles (PAVs) that are as safe, reliable, cost effective, and easy to use as cars would give us 
tremendous new freedom to travel for work and play. While the idea of "everyman's" airplane is as old as aviation, 
the technology to achieve this dream is close at hand. That said, the technology of contemporary, highly-automated 
aircraft and emerging autonomous vehicles, by itself, is unlikely to enable passenger-carrying PAVs in the near 
term. At first glance, these technologies appear to make full-autonomy seem likely in the near future. However at 
present, even emerging autonomous vehicles rely on specialized, task-specific algorithms that have limited ability 
respond appropriately in novel situations. Since the environmental and situational complexity of flight virtually 
assures novel situations will occasionally be encountered, fully autonomous airplanes are probably a distant vision, 
at least for safety-critical applications like carry human cargo. Further, the conservative nature of aviation 
regulation and regulators makes it unlikely that technology with sufficient intelligence for fully -autonomous 
operation could be certified for safety-critical applications without first having significant operational history in less 
critical applications. These factors lead us to conclude the only credible path to PAVs must retain the human in the 
pilot’s seat and leverage their perception, common sense, general intelligence, and creative thinking. At the same 
time to enable ease-of-use with improved safety, technology must be used to perform or support tasks requiring 
swift and precise control, extreme vigilance, extensive memory, and complex computation. 

We have applied this philosophy to the development of a haptic -multimodal flight control (HFC) system that 
supports the pilot in the conduct of control and tactical trajectory management tasks in a way that is easy to learn 
and use. The HFC provides protection against the most common contributors to accidents and is designed to avoid 
or mitigate human-automation interaction breakdowns that are common contributors to loss of situation awareness 
on current highly automated aircraft. The HFC differs from conventional automation in several important ways. 
First, all control automation is managed and monitored through the primary controls and displays. The pilot skills 
and actions used to direct the automation are essentially the same as used for assisted-manual flight and workload 
peaks are avoided by not requiring separate autopilot programming and data entry tasks. Finally, unlike current 
control automation, the HFC is designed to require regular, but simple, physical involvement by the pilot during the 
conduct of a flight. 

Initial simulation evaluations of the HFC support claims that it is easy to learn and use and that this ease-of-use 
is largely retained in turbulent flight conditions. In a comparison with manual and fully -automated forms of 
interaction, it was preferred by a large majority of test subjects and tended to increase subjective awareness ratings 
of the flying task and the overall set of tasks. 

Significant work remains before the full HFC could be applied to PAVs. The most technically uncertain aspect 
is the reliance on full-time, stability and control augmentation to provide simplified flight dynamics and envelope 
protection. Flight testing is needed to verify that ease-of-use suggested in simulation is achieved in flight, 
particularly during takeoff and landing. Also, design and certification criteria are needed for a practical 
implementation retaining simplified dynamics in the presence of subsystem and component failures not shown to be 
extremely improbable. Other aspects of the HFC, in particular the integration of automation management into the 
primary controls and displays, could be applied to current, highly -automated aircraft and have the potential to 
significantly improve the safety and usability of these aircraft. 
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